The Design and Implementation of a MPI-Based Parallel File System
نویسندگان
چکیده
This paper presents the design of an MPI (Message Passing Interface)-based parallel file system, MPFS. MPI-IO is an extension of MPI which supports flexible logical file partition and physical file organization as well as a rich set of file access functions. MPFS enables users to specify both logical file partitions among user processes and physical file data layouts across data servers. The present implementation includes full support of file data distribution, asynchronous I/O, shared file pointers, collective I/O operations, and limited support of data layout hints. Moreover, MPFS achieves portability through a user library that conforms to a generic UNIX interface. However, MPFS does not support the FORTRAN interface, error handling or profiling in the MPI-IO specification. MPFS has been implemented on a workstation cluster connected by Fast Ethernet. The performance measurements of MPFS show that the message passing overhead is minimal, and that the performance is limited by the bandwidth of Fast
منابع مشابه
Parallel computing using MPI and OpenMP on self-configured platform, UMZHPC.
Parallel computing is a topic of interest for a broad scientific community since it facilitates many time-consuming algorithms in different application domains.In this paper, we introduce a novel platform for parallel computing by using MPI and OpenMP programming languages based on set of networked PCs. UMZHPC is a free Linux-based parallel computing infrastructure that has been developed to cr...
متن کاملParleda: a Library for Parallel Processing in Computational Geometry Applications
ParLeda is a software library that provides the basic primitives needed for parallel implementation of computational geometry applications. It can also be used in implementing a parallel application that uses geometric data structures. The parallel model that we use is based on a new heterogeneous parallel model named HBSP, which is based on BSP and is introduced here. ParLeda uses two main lib...
متن کاملA Windows-Based Parallel File System
Parallel file systems are widely used in clusters to provide high performance I/O. However, most of the existing parallel file systems are based on UNIX-like operating systems. We use the Microsoft .NET framework to implement a parallel file system for Windows. We also implement a file system driver to support existing applications written with Win32 APIs. In addition, a preliminary MPI-IO libr...
متن کاملHint Controlled Distribution with Parallel File Systems
The performance of scientific parallel programs with high file-I/O-activity running on top of cluster computers strongly depends on the qualitative and quantitative characteristics of the requested I/Oaccesses. It also depends on the corresponding mechanisms and policies being used at the parallel file system level. This paper presents the motivation and design of a set of MPI-IO-hints. These h...
متن کاملA scalable MPI implementation of an ad-hoc parallel I/O system
This paper presents the novel design, implementation and evaluation of an ad-hoc parallel I/O system (AHPIOS). AHPIOS is the first scalable parallel I/O system completely implemented in Message Passing Interface (MPI). The MPI implementation brings the advantages of portability, scalability and high-performance. AHPIOS allows MPI applications to dynamically manage and scale distributed partitio...
متن کامل